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(57) A watermark data insertion method for insert- 
ing watermark data into an input original signal is pro- 
vided. The method includes: a frequency transform step 
for applying a frequency transform to the original signal 
to form an intermediate signal; a watermark data inser- 
tion stop for inserting the watermark data into a first set 
of frequency component of the intermediate signal; and 
a frequency inverse transform stop for applying an 
inverse frequency transform to the intermediate signal, 
the watermark data having been inserted into the inter- 

FIG. 1 



mediate signal, thereby obtaining a signal containing 
the watermark data embedded therein. The watermark 
data insertion step includes: determining, based on 
pseudo random numbers, the first set of frequency com- 
ponents of the intermediate signal into which the water- 
mark data is inserted; and using a second set of 
frequency conponents of the intermediate signal as a 
reference signal, the watermark data not being inserted 
into the second set of frequency components. 
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Description 

BACKGROUND OF THE INVENTION 

1. FIELD OF THE INVENTION: 

The present invention relates to a watermark dots 
Insertion method for inserting watermark data into a 
drgrtalized signal representing images, sounds, and/or 
the like to indicate their copyrights, and a watermark 
data detection method for detecting inserted watermark 
data. 

2. DESCRIPTION OF THE RELATED ART: 

Techniques for inserting watermark data into a digi- 
talized signal representing images, sounds, and/or the 
like to indicate their copyrights having been vigorously 
studied and developed. Such watermark data is 
required to have minimum interference with the images 
and/or sounds of interest, be hardly visit»le (in the case 
of image data) and hardly audible (in the cane of sound 
data), and yet be easily detected by a person knowing 
the detection method while preventing unauthorized 
removal of the watermark data by third parties. By using 
such watermark data, for example, infringement of a 
copyright (by illegal copying of images and/or sounds) 
by unauthorized third parties can be established or pre- 
vented. 

One conventional method for inserting watermark 
data into an original signal representing images, 
sounds, and/or the like utilizes a spread spectrum tech- 
nology. In principle; this method involves: multiplying the 
original signal representing images, sounds, and/or the 
like by random numbers to obtain a uniform spectrum; 
adding watermark data to the multiplied signal; and 
again multiplying the resultant signal writh the same ran- 
dom numbers to restore the original signal. At the time 
of restoring the original signal, the added watermark 
data is also multiplied by the random numbers. As a 
result, the watermark data has a uniformly spread spec- 
trum, i.e. becomes white noise, in the restored original 
signal. 

Figure 7 is a block diagram illustrating the structure 
of a conventional watermark data insertion apparatus 
utilizing a spread spectrum technology. As shown in Fig- 
ure 7, an original signal 3; (into which watermark data Xj 
is to be inserted) is input to a multiplier 52, where it is 
multiplied by predetermined pseudo random numbers 
to become aiXC|. Since the pseudo random numbers Cj 
are a random sequence of +1 and -1. the frequency 
spectrum of a|xCi becomes uniform. 

Next, the watermark data Xj is input to an adder 53 
to be added to a DC component of the above multiplica- 
tion product a|XC|. Although not shown, the adder 53 
includes a cascade connection of a Fourier transformer, 
an adder, and an inverse Fourier transformer for extract- 
ing a DC component of an input signal. 



The signal ajXCj + Xj (i.e., the randomized aiginal 
signal plus the watermark data) is input to a multiplier 
54. where it is again multiplied by the same pseudo ran- 
dom numbers to give a^xc-^ + XjXCi. Since the pseudo 

5 random numbers are +1 or -1 , Cj^ equals 1 . Therefore, a 
signal having a value of a| + x^xC) (= ajXCi^ + X|XCi) is 
obtained, i.e., a signal with embedded watermark data 
Rgure 8 is a block diagram illustrating the structure 
of a conventional watermark data detection apparatus 

10 utilizing a spread spectrum technology for detecting 
embedded watermark data. As shown in Figure 8, a sig- 
nal with embedded watermark data 3) + X|XCi is input to 
a multiplier 62. where it ii multiplied by predetermined 
pseudo random numbers to become 3p<C| + X|XC|^ = 

75 3jXC{ + X|. Since the pseudo random numbers ci are a 
uniform random sequence of +1 and -1, the DC compo- 
nent of aixci becomes zero, so that only the watermark 
data X{ is extracted by an extractor 63. The extractor 63 
includes a Fourier transformer for extracting a DC com- 

20 ponent of an input signal. 

The above-described technique is reported in 
"Watermark data signature method for images utilizing 
PN series". Electronic Information Communication 
Society, "Encryption and Information Security Sympo- 

25 sium" proceedings, SCIS '97-26B, 1997. 

However, ttie above-described conventional tech- 
nique has a disadvantage in tiiat the watermark data 
may become noticeable where tiie original signal has a 
small amplitude because the watern^rk data is inserted 

30 into the original signal in a uniform manner, i.e., regard- 
less of the characteristics of the original signal. There is 
also a disadvantage in that if the amplitude of the 
watermark data is decreased in order to prevent tiie 
watermark data from becoming noticeable, the water- 

35 mark data may be masked by the original signal and 
therefore difficult to detect. Furthernwre. unauthorized 
third parties can make the waternnark data impossible to 
detect by intentionally adding noise or the like to the 
original signal. 

40 

SUMMARY OF THE INVENTION 

A watermark data insertion method for inserting 
watermark data into an input original signal according to 

45 the present invention includes: a frequency transform 
step for applying a frequency transform to the aiginal 
signal to form an intermediate signal; a watermark data 
insertion step for inserting the watermark data into a 
first set of frequency connponents of the intermediate 

50 signal; and a frequency inverse transform step for 
applying an inverse frequency transform to the interme- 
diate signal, the watermark data having been inserted 
into the intermediate signal, thereby obtaining a signal 
containing tiie watermark data embedded therein. 

55 wherein the watermark data insertion step includes: 
det rmining. based on pseudo random numbers, the 
first set of frequency components of the intermediate 
signal into which tiie watermark data in inserted; and 
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using a second set of frequency components of ttie 
intermediate signal as a reference signal, the water- 
mark data not being inserted into the second set of fre- 
quency components. 

In one embodiment of the invention, the frequency 5 
transform applied in the frequency transform step is a 
wavelet transform, and the inverse frequency transform 
applied in the frequency inverse transform step is an 
inverse wavelet transform. 

In another embodiment of the invention, the water- 
mark data insertion step includes: a weighting step for 
increasing or decreasing a signal amplitude of the 
watermark data in accordance with an amplitude of the 
first set of frequency components of the intermediate 
signal into which the watermark data is inserted. 

In still another embodiment of the invention, the 
watermark data insertion step Includes determining the 
first set of frequency components of the intermediate 
signal into which the watermark data is inserted by 
using pseudo random numbers having frequency- 
dependent weights. 

In still another embodiment of the invention, the 
watermark data insertion step includes: a weighting 
step for increasing or decreasing a signal amplitude of 
the watermark data in accordance with a frequency of 
the first sat of frequency components of the intermedi- 
ate signal into which the watermark data is inserted. 

In still another embodiment of the invention, the 
watermark data insertion step includes converting 
binary data representing the watermark data into prede- 
termined positive and negative values, and thereafter 
inserting the converted binary data into the first set of 
frequency components of the intermediate signal. 

Ahernatively, the watermark data insertion method 
for inserting watermark data into an input original signal 
according to the present invention includes: determining 
a position at which the watermark data is to be inserted 
in accordance with pseudo random numbers which 
have been subjected to an inverse transform; subjecting 
the watermark data to an inverse transform; increasing 
or deaeasing a signal amplitude of the inverse-trans- 
formed watern^rk data in accordance with an ampli- 
tude of the original signal; and inserting the inverse- 
transformed watermark data with the increased or 
decreased signal amplitude into the original signal. 

A watermark data detection method for detecting 
watermark data within a signal containing embedded 
watermark data according to the present invention 
includes: an inverse-transformed pseudo random 
number generation stop for applying an inverse fre- 
quency transform to pseudo random numbers used for 
determining an insertion position of the watermark data 
and complements of the pseudo random numbers, 
respectively; a multiplication step for multiplying the sig- 
nal containing embedded watermark data by the values 
generated in th inverse-transformed pseudo random 
number generation step t obtain signal components to 
which the watermark data has been added and signal 



components to which the watermark data has not been 
added; and an averaging stop for deriving a difference 
between an average of the signal components to which 
the waternnark data has been added and an average of 
the signal components to which the watermark data has 
not been added as obtained in the multiplication step, 
whereby the difference is detected as the watermark 
data. 

In one emtxxliment of the invention, the averaging 
step includes deriving the average of the signal conpo- 
nents to which the watermark data has been added by 
applying a weight to an average of high frequency com- 
ponents among the signal components to which the 
watermark data has been added, the weight being 
smaller than weights applied to averages of frequency 
components other than the high frequency components. 

Alternatively, the watermark data detection method 
for detecting watermark data within a signal containing 
embedded watermark data according to the present 
invention includes: a frequency transform stop for apply- 
ing a frequency transform to the signal containing 
enobedded watern^rk data to form an intermediate sig- 
nal; and a watermark data extraction step for extracting 
the watermark data from a predetermined frequency 
component of the intermediate signal. 

In one embodiment of the invention, the watermark 
data extraction stop includes determining the predeter- 
mined frequency component of the intermediate signal 
from which the watermark data is extracted based on 
pseudo random numbers. 

A watermark data insertion apparatus for inserting 
watermark data into an input original signal according to 
the present invention includes: a frequency transform 
section for applying a frequency transform to the origi- 
nal signal to form an intermediate signal; a watermark 
data insertion section for inserting the watermark data 
into a first set of frequency components of the interme- 
diate signal; and a frequency inverse transform section 
for applying an inverse frequency transform to the inter- 
mediate signal, the watermark data having been 
inserted into the intermediate signal, thereby obtaining 
a signal containing the watermark data embedded 
tiierein, wherein the watermark data insertion section 
determines the first set of frequency components of the 
intermediate signal into which the watermark data is 
inserted based on pseudo random numbers, and uses a 
second set of frequency components of the intermedi- 
ate signal as a reference signal, the watermark data not 
being inserted into the second set of frequency compo- 
nents. 

A waternnark date detection apparatus for detecting 
watermark date witiiin a signal containing embedded 
watermark data includes: an inverse-transformed 
pseudo random number generation section for applying 
an inverse frequency ti-ansform to pseudo random num- 
bers used for determining an insertion position of the 
watermark data and complements of the Pseudo ran- 
dom numbers, respectively; a multiplication section for 



IS 



20 



25 



30 



35 



40 



45 



50 



3 



5 

multiplying the signal containing embedded watermark 
data by the values generated in the inverse-transformed 
pseudo random nunt>er generation section to obtain 
signal components to which the watermark data has 
been added and signal components to which the water- 5 
mark data has not been added; and an averaging sec- 
tion for deriving a difference between an average of the 
signal components to which the watermark data has 
been added and an average of the signal components 
to which the watermark date has not been added as 10 
obtained by the multiplication section, whereby the dif- 
ference is detected as the watermark data. 

In accordance with the above-desaibed method: 
and apparatuses of the present invention, the insertion 
position and the signal amplitude of watermark data are is 
varied in accordance with the frequency components 
and amplitude of an original signal, thereby making it 
difficult to detect the watermark data. This n\akes possi- 
ble the insertion of watermark data that has a large 
amplitude, thereby preventing the destruction of water- 20 
mark data by an intentional addition of noise or the like. 
Since the insertion position of the watermark data is 
determined by utilizing pseudo rarxiom numbers, those 
who do not know the insertion position cannot read the 
watermark data, thereby preventing unauthorized tam- 2s 
paring with the watermark data. Furthermore, the ran- 
dom numbers used for determining the insertion 
posKion of the watermark data are visually weighted so 
as to render the perception of the watermark data even 
more difficult. 30 

Thus, the invention described heroin makes possi- 
ble the advantage of provkJing a watermark data inser- 
tion method and a watermark data detection method for 
inserting watermark data in such a manner that the 
watermark data cannot be easily detected and yet 35 
imposes minimum interference with an original signal, 
such that the watermark data is hardly visible and/or 
audible and difficult to remove by unauthorized means 
and yet easily detectable through an authorized or 
intended procedure. 40 

This and other advantages of the present invention 
will become apparent to those skilled in the art upon 
reading and understanding the following detailed 
description with reference to the accompanying figures. 

45 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a block diagram illustrating the structure 
of a watermark data insertion apparatus according 
to one embodiment of the watermark data insertion so 
method of the present invention. 

Figure 2 is a diagram showing the data arrange- 
ment of transformed coefficients of an image which 
has been broken down into two-dimensional fre- ss 
quency conponents by wavelet transform. 

Figure 3 is a diagram showing the position at which 
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watermark date is embedded in an original signal 
coordinate system corresponding to the insertion of 
watermark data into a subband i in Figure 2. 

Figure 4 in a block diagram illustrating the structure 
of a watermark data insertion apparatus as another 
embodiment of the watermark data insertion 
method according to the present invention. 

Figure 5 is a block diagram illustrating the structure 
of a watermark data detection apparatus according 
to one embodiment of the watermark data detection 
metiiod of the present invention. 

Rgure 6 is a block diagram illustrating the structure 
of a watermark data detection apparatus as 
another embodiment of the watermark data detec- 
tion method according to tiie present invention. 

Rgure 7 is a block diagram illustrating the structure 
of a conventional watermark data insertion appara- 
tus utilizing a spread spectrum technology 

Rgure 8 is a block diagram illustrating the structure 
of a conventional watermark data detection appara- 
tus utilizing a spread spectrum technology 

DESCRIPTION OF THE PREFERRED EMBODI- 
MENTS 

Hereinafter, an example of tiie watermark data 
insertion method according to the present invention will 
be described with reference to the accompanying fig- 
ures. Figure 1 is a block diagram illustrating the struc- 
ture Of a watermark data insertion apparatus according 
to one embodiment of the watermark data insertion 
method of the present invention. The watermark data 
insertion apparatus includes a transfomri section 1 , a 
watermark data insertion section 2, and an inverse 
transform section 3. 

The transform section 1 , which is mainly composed 
of a wavelet ti^ansform section 11 in the illustrated 
embodiment, breaks down an input image signal into 
two<Jimensional frequency components. The wavelet 
transform section 11 includes a pair of filters, i.e., a 
high-pass filter and a low-pass filter (not shown), for 
separating an image signal into a high frequency com- 
ponent and a low frequency component, respectively 
By repetitively passing the resultant high and low com- 
ponents tiirough the same pair of filters, the wavelet 
b^ansform section 11 breaks down the image signal into 
a large number of two-dimensional frequency compo- 
nents. A simplest wavelet ti'ansform section can be 
implemented by using a low-pass filter having multipli- 
cation coefficients (1/2, 1/2) and a high-pass filter hav- 
ing multiplication coefficients (1/2,-1/2) and discarding 
ev ry otiier sample signal of the low frequency compo- 
nents and tiie high frequency components which 
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passed through the respective filters. 

Figure 2 is a diagram showing the data arrange- 
ment of transformed coefficients of an image which has 
been broken down into two-dimensional frequency com- 
ponents by wavelet transform. By a first-level wavelet 
transform, transformed coefficients in regions (sub- 
bands) LL. LH. HL. and HH, in each of which the 
number of transformed coefficients is reduced to 1/2 
along both the vertical and horizontal dimensions, are 
provided. For example, the sut>band LL consists of 
transformed coefficients which can be obtained by dis- 
carding every other pixel along both the vertical and 
horizontal dimensions. Therefore, restoring an image 
using the transformed coefficients from this subband 
results in an image having a 1/2 resolution of that of the 
original image. The subband LL consists of transformed 
coefficients which are derived from image components 
existing in the lower half of the frequency spectrum 
along both the vertical and horizontal dimensions. The 
subband HH consists of transformed coefficients which 
are derived from image components existing in the 
higher half of the frequency spectrum along both the 
vertical and horizontal dimensions. The subband HL 
consists of transformed coefficients which are derived 
from image components existing in the lower half along 
the vertical dimension, and in the higher half along the 
horizontal dimension, of the frequency spectrum. Simi- 
larly, the subband LH consists of transformed coeffi- 
cients which are derived from image connponents 
existing in the higher haH along the vertical dimension, 
and in the lower half along the horizontal dimension, of 
the frequency spectrum. 

By further breaking down an image represented by 
the subband LL into higher and lower frequency compo- 
nents, subbands are obtained, each of which can be 
represented by a smaller number of transformed coeffi- 
cients. Hereinafter, the transformed coefficients (fre- 
quency components) of a subbarxl i will be denoted as 
Bj (intermediate signal), where i is a natural number. It 
will be appreciated that the wavelet transform is simi- 
larly applicable to audio data as well. 

The watermark data insertion section 2 shown in 
Figure 1 includes a pseudo random number generation 
section 21 , a multiplication section 22, a weighting/addi- 
tion section 23, and a subtraction section 24. 

The pseudo rardom number generation section 21 
generates pseudo random numbers PN and outputs a 
signal PNy, which is weighted by the pseudo random 
numbers PN, by employing an internal audio or visual 
filter (not shown). The multiplication section 22 multi- 
plies the output B| of the transform section 1 by the out- 
put PNy of the pseudo random number generation 
section 21 so as to output a multiplication product bj = 
BixPNi|. 

The weighting/addition section 23 receives water- 
mark data WM| (repr senting the actual watermark), 
which is converted into watermark data Wj weighted by 
weighting coefficients corresponding to tiie respective 



subbands. The weighting/addition section 23 adds the 
watermark data Wj and the output signal b| of tiie multi- 
plication section 22 so as to output a signal X{ = b] + Wj. 
The subtraction section 24 sutstracts the output signal b| 

5 of tiie multiplication section 22 from tiie output Bj of tiie 
transform section 1 so as to output the subtraction result 
fj. The inverse transform section 3 includes an inverse 
wavelet transform section 31 for receiving the, signals Xj 
and T\ and transforming tiiese signals into an image sig- 

10 nal with embedded watermark data, which is output 
from the inverse transform section 3. 

The operation of the watermark data insertion 
apparntus having the atx>ve-desaibed structure will be 
described in more detail. A digitaltzed image signal is 

15 transformed into signals B| of each subband i (where I 
is a natural number) by the wavelet transform section 
11. The sensitivity of human eyes or ears becomes rel- 
atively high for the case where tiie signals Bj of a sub- 
band represent the components of a middle frequency 

20 band, and relative low for tiie case where the signals Bj 
of a subband represent the components of a low or high 
frequency band. The pseudo random number genera- 
tion section 21 and the weighting/addition section 23 
subject each subband to a weighting in accordance witii 

25 such characteristics of tiie human visual or auditory 
sense. 

The pseudo random number generation section 
generates pseudo random numbers PN consisting of 0 
or 1. It will be appreciated that by employing these 
30 pseudo random numbers PN as they are, it would result 
in the multiplication section 22 selecting all tiie sub- 
bands in an impartial or uniform manner. 

Ratiier. in order to prevent the unauthorized detec- 
tion of the insertion position of the watermark data 
35 WMjj. the uniform pseudo random numbers PN are con- 
verted into pseudo random numbers having the oppo- 
site characteristics of tiiose of tiie human visual or 
auditory sense (frequency sensitivity), by being passed 
through a filter for cutting off the components of the mid- 
40 die frequency band, for example. The pseudo random 
number generation section 21 converts the ps^do ran- 
dom number PN into pseudo random numbers PNjj 
which are weighted by means of a frequency weighting 
filter (not shown). The use of such pseudo random num- 
45 bers PNjj advantageously changes tiie random pattern 
from a white noise-like pattern into a pink noise-like pat- 
tern, which has less middle frequency components, 
hence making it difficult to detect. 

The multiplication section 22 multiplies the signals 
50 Bj of each subband by the predetermined pseudo ran- 
dom numbers PNjj to output a multiplication product sig- 
nal bj (i.e., bj = Bj X PNjj). Thus, at least one of tiie 
t-ansformed coefficients of each subband that is multi- 
plied by bit 1 of the pseudo random numbers PNjj is 
55 extracted so as to be output as the signal bj. 

Upon receiving the watermark data WI\Aj (of binary 
values of 0 or 1), the weighting/addition section 23 con- 
verts each value of the watermark data WMg into eith r 
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a positive predetermined value (when the value of the 
watermark data WMj is 1) or a negative predetermined 
value (when the value of th water n^rk data WMj is 0). 
Furthermore, th weighting/addition section 23 multi- 
plies the transformed watermark data by a weight Kjj 5 
corresponding to the subband from which the signals b] 
have been extracted so as to output the mullipiication 
product as the watermark data Wj. One of respectively 
different values previously assigned to the respective 
sut)bands is used as the weight Kjj for each subband i. to 

Specifically, the value of the weight K{j is decreased 
in the middle frequency band, where humans have high 
frequency sensitivity. That is, the values of the weight 
K{j are selected to realize the opposite characteristics of 
those of the hunnan visual or auditory sense. As a result, is 
even If the watermark data Wj is inserted in a subband 
containing middle frequency band components, the 
watermark data Wj is hardly visible (in the case of an 
image signal) and hardly audible (in the case of an 
audio signal). 20 

By using the pseudo random numt^ers PNy thus- 
weighted by the use of the frequency weighting filter 
and the weight K^j having values corresponding to the 
respective subbands, the watermark data Wj will none- 
theless be inserted in all subbands as a result of 25 
processing a series of image signals representing a plu- 
rality of pictures. Therefore, even if tampering is 
attempted by passing an Image signal containing 
embedded watermark data through a low-pass filter or 
removing a portion of the signal, the watermark data will 30 
remain in the transformed coefficients In the middle and 
low frequency bands and therefore be detectable 
despite such tampering. 

The weighting/addition section 23 adds the water- 
mark data Wj to the signal bi and outputs a signal X| (= 35 
bj + Wp containing embedded watermark data. The sig- 
nal amplitude of the watermark data Wj is selected to be 
smaller than that of the signal b| by increasing or 
decreasing the signal amplitude of the watermark data 
Wj in such a manner that the watermark data Is masked 40 
by the signal b\ to prevent unauthorized detection. 

The subtraction section 24 calculates a signal ri (= 
Bj - bj) into which the watermark data was not inserted, 
and supplies the signal r| to the inverse transform sec- 
tion 3. The inverse transform section 3 synthesizes the 45 
signals Xi and r| and performs a transform process at an 
inverse wavelet transform section 31 for restoring the 
original Image signal. As a result of thus restoring the 
image, an image signal with embedded watermark data 
is obtained. so 

The inverse wavelet transform can be achieved by. 
Interpolating 0 between sample signals of each sub- 
band, passing the Interpolated sample signals through 
the same high-pass fitter and low-pass filter that were 
used for the wavelet transform, and repetitively deriving 55 
a difference between each pair of a low frequency com- 
ponent and a high frequency component output from 
the high -pass filter and low-pass filter. 
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The abov -described process is repeated for the 
same watermark data, whereby Image signals repre- 
senting images containing the same watermark data 
embedded therein are sequentially output. Atternatlvely, 
the addition of the same watermark data Wj to the sig- 
nal bj Is performed multiple times so that a plurality of 
instances of watermark data are Inserted in the signal 
Xj. In either case, a plurality of instances of the same 
watermark data will always be Inserted into the image 
signal. 

Rgure 3 Is a diagram showing the position at which 
watermark data is embedded in an original signal coor- 
dinate system as restored by an inverse transform 
based on the simplest wavelet transform, assuming that 
tiie watermark data was embedded at a position X| 
among the transformed coefficients Bj of a suk)band I 
(that lies In a region of high frequency components 
along botii the horizontal direction and tiie vertical 
direction of the subband LL in Figure 2). The signal 
amplitude of the watermark data Is reduced to 1/1 6 due 
to the Inverse transform, and its polarity (i.e.. positive or 
negative) Is also changed, so tiiat the inverse-trans- 
formed pseudo random numbers do not take binary val- 
ues but may take any real number between -1 and +1 . 

Rgure 4 is a block diagram illustrating the structure 
of a watermark data Insertion apparatus as another 
embodiment of the watermark data Insertion method 
according to the present invention. As shown in Figure 
4, a multiplication section 72 directly multiplies an Input 
Image signal Aj by Inverse-t-ansformed pseudo random 
numbers which are generated by an inverse-trans- 
formed pseudo random number generation section 71 
so as to form a sample Bj (i.e.. a predetermined fre- 
quency component of an intermediate signal), which is 
output so that watermark data will be subsequentiy 
added tiiereto. Watermark data WMj is transformed into 
watermark data Wj In a base band region by an inverse 
transform section 73, and the transformed watermark 
data Wj is input to a weighting/addition section 74. The 
weighting/addition section 74 adds the transformed 
watermark data Wj and the sample B\ so as to output a 
signal a{ + Wj containing embedded waternnark data. A 
subtraction section 75 calculates a sample Af - B\ In 
which the watermark data is not embedded. An addition 
section 76 adds tiie output of the weighting/addition 
section 74 and the output of the substraction section 75 
so as to output an Image signal containing embedded 
watermark data A| + Wj. 

In accordance with the above apparatus, an inverse 
transform of pseudo random numbers (indicating posi- 
tions at which watermark data Is to be inserted) based 
on an Inverse wavelet transform, etc., is previously cal- 
culated. TTius. the Inverse-fransfbrmed pseudo random 
numbers can be used to multiply an input image signal 
directly. I.e.. without subjecting tiie image signal to a 
transform process. Moreover, it becomes unnecessary 
to perform an inverse transform after the addition of 
waternriark data. As a result, the cost associated with 
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the insertion of watermark data can be reduced. 

Next, one embodiment of the watermark data 
detection method according to the present inv ntion will 
be desatbed. Rgure 5 is a block diagram illustrating the 
structure of a watermark data detection apparatus s 
according to one embodiment of the watermark data 
detection method of the present invention. The water- 
mark data detection apparatus includes a watermark 
data detection section 4, which in turn includes an 
inverse-transformed pseudo random number genera- w 
Won section 41 , a multiplication section 42, and an aver- 
aging section 43. 

The inverse-transformed pseudo random number 
generation section 41 includes a pseudo random 
number generation section, which is identical with that is 
in the watermark data insertion apparatus shown in Fig- 
ure 1 , for generating pseudo random numbers PNy and 
(1 - PNiP and inverse transforming the pseudo random 
numbers PNy and (1 - PNy) by inverse wavelet trans- 
form to generate and output inverse-transformed 20 
pseudo random numbers PN[j' and (1 - PNiP'. In Figure 
5, the inverse-transformed pseudo random numbers 
PNjj' and (1 - PNjj)' are shown as Nj and Mj^ respec- 
tively; Nj is an inverse wavelet transform of the pseudo 
random numbers PNjj used for the determination of 25 
watermark data insertion position; and M; is an inverse 
wavelet transform of the complement of the pseudo ran- 
dom numbers PNg. 

Upon receiving an image signal P containing 
embedded watermark dots, the multiplication section 42 so 
multiplies the image signal P by the inverse-transformed 
pseudo random numbers Nj and M| to output a signal X|' 
(= P X PNjp and a signal t{ (= P x (1 - PUy^'). 

By using these inverse-transformed pseudo ran- 
dom numbers N; and M|, it becomes unnecessary to 35 
subject image data containing watermark data to a 
wavelet transform in order to achieve detection of the 
watermark date. As a result, the time and cost required 
for detection are reduced. 

The resultant signal Xj' is a signal containing 40 
embedded watermark data. The resultant signal r|' is a 
signal not containing the watermark data embedded 
therein. 

The signals X|* and r|' are input to an averaging sec- 
tion 43, where a cumulative sum Lxi' and a cumulative 4S 
sum £rr are calculated for the signals xf and Tj*, respec- 
tively. A difference therebetween (Lxi* • Lri*) is calcu- 
lated. 

The value of the above difference equals X|'- r|' = 
{b{ + WMj') - fj*. Since the signals b| are generated by 50 
nondiscriminatory selection based on random numbers, 
an average of a multitude of and an average of a mul- 
titude of r|* will satisfy bj' = ti (because the population of 
bj' and the population of Ti are identical). Thus, the 
above subtraction leaves WMj* (= (b|* + WMj') - f{), i.e., ss 
the watermark data. Based on the signs (i.e. . positiv or 
negative) of this value, the value of the watermark data 
WMj can be obtained. 



When calculating an average of the signals X|* con- 
taining emt>edded watermark data, the average of 
higher frequency components receives a weight which 
is smaller than the weights for other average values. As 
a result, even if an unauthorized third party attempts to 
destroy or render undetectable the watermark data, or if 
the size or the position of the signal containing embed- 
ded watermark data is changed, or if noise is admixed in 
the high frequency components, such influences can be 
effectively eliminated. 

Rgure 6 is a block diagram illustrating the structure 
of a watermark data detection apparatus as another 
emkxxliment of the watermark data detection method 
according to the present invention. As shown in Figure 
6, a transform section 81 receives an image signal P 
containing embedded watermark data and performs a 
wavelet transform to transform the image signal P into 
transformed coefficients of each subband. This process 
is in the opposite direction with respect to the process 
performed by the inverse transform section 3 in Figure 
1. The signal (i.e., intermediate signal) which results 
from this process is equal to Xj + in the emtxxiiment 
described with reference to Figure 1 . A multiplication 
section 83 multiplies the signal X} + r| by pseudo random 
numbers generated by a pseudo random number gen- 
eration section 82 (which are the same pseudo random 
numbers as those used in the embodiment described 
with reference to Figure 1) so as to extract a signal X| 
containing embedded watermark data. The signal Xj 
consists of the original signal bj with tiie watermark data 
W| added tiiereto. The averaging section 85 derives an 
average of a plurality of signals Xj containing embedded 
watermark data to give a sum (B 4 W) of an average B 
of the original signals b| and an average W of the water- 
nrtark data. 

Only the signal ri (= Bg - bj) in which the watermark 
data was not embedded is output from a subtraction 
section 84. An averaging section 86 derives on average 
of a plurality of signals r|, thereby giving an average B of 
the original signals. A subti'action section 87 subtracts 
tiie average B from the average containing tiie water- 
mark data (B + W). thereby obtaining the watermark 
data W. 

Since the input signal is subjected to wavelet trans- 
form in tfie apparatus shown in Rgure 6, the same 
pseudo random numbers as those used in the above- 
described watermark data insertion apparatuses can be 
used. 

Although a wavelet transform section is exemplified 
as a transform section in the above-illustrated errdxxii- 
ments. the some effects will be provided by employing a 
frequency ti-ansfbrm means such as Hadamard trans- 
form, discrete cosine transform, Fourier trarisform, Kar- 
hunen-Lo§ve transform, or tiie like. Pseudo random 
numbers can be generated by using linear feedback 
shift registers. Alternatively, previously calculated 
pseudo random numbers can be stored in a memory or 
tiie like for use in the process of tiie present invention. 
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Various sections illustrated in the enfibodtments can 
be implemented as computer progranr^ or dedicated 
digital circuits. 

As described above, the watermark data insertion 
method and watermark data detection method accord- s 
ing to the present invention make it possible to insert 
watermark data in a manner such that the watermark 
data is very difficult to detect by unauthorized third par- 
ties while being easily detectible by an authorized per- 
son, e.g.. one who inserted the watermark data. Thus, 10 
the present invention makes possible the establishment 
of infringement of a copyright of copyrightable works 
and/or prevention of illegal copying of such works in the 
field of digital contents containing images and/or 
sounds. 15 

In accordance with one embodiment of the present 
invention, a wavelet transform encompasses the pixel 
information of the entire single frame regardless of the 
subband that is being used. Therefore, watermark data 
can be easily embedded in the entire image through 20 
one watermark data insertion process. The method of 
the present invention also prevents block distortion, 
which is typically observed in DCT transform. 

In accordance with one embodiment of the present 
invention, watermark data is inserted with its amplitude 25 
being adjusted in accordance with the luminance values 
of pixels, whereby the watermark data can be better 
concealed within the image. 

In accordance with one embodiment of the present 
invention, watermark data can be inserted in positions so 
in the pixel frequency space where the watermark data 
will not be easily noticeable. 

In accordance with one embodiment of the present 
invention, the DC component of the watermarked data 
is not changed after insertion, thereby improving the 35 
signal to noise ratio in tiie averaging process during 
detection. 

In accordance with one embodiment of the present 
invention, it is unnecessary to detect watermark data by 
refemng to a reference image, tiiereby reducing the bur- 40 
den on the detection system. 

In accordance with one embodiment of the present 
invention, watermark data can be detected even after 
processing a signal vtnth a high-cut filter. 

Various other modifications will be apparent to and 4s 
can be readily made by those skilled in the art without 
departing from tiie scope and spirit of this invention. 
Accordingly, it is not intended tiiat the scope of the 
claims appended hereto be limited to the description as 
set forth herein, but rather that the claims be broadly so 
construed. 

Claims 

1. A watermark data insertion method for inserting 55 
watermark data into an input original signal, com- 
prising: 
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a frequency transform step for applying a fre- 
quency transform to the original signal to form 
an intermediate signal; 

a watermark data insertion stop for inserting 
the watermark data into a first set of frequency 
components of the intermediate signal; and 
a frequency inverse transform step for applying 
an inverse frequency transform to the interme- 
diate signal, the watermark data having been 
inserted into the intermediate signal, thereby 
obtaining a signal containing the watermark 
data embedded therein, 
wherein the watermark data insertion step 
comprises: 

determining, based on pseudo random 
numbers, the first set of frequency compo- 
nents of the intermediate signal into which 
the watermark data is inserted; and 
using a second set of frequency compo- 
nents of the intermediate signal as a refer- 
ence signal, tiie watermark data not being 
inserted into tiie second set of frequency 
components. 

2. A watermark data insertion method according to 
daim 1 , wherein the frequency transform applied in 
tiie frequency transform step is a wavelet trans- 
form, 

and the inverse frequency transform applied 
in the frequency inverse transform step is an 
inverse wavelet transform. 

3. A watermark date insertion method according to 
claim 1, wherein tiie watermark data insertion step 
comprises: 

a weighting step for inaeasing a decreasing a 
signal amplitude of tiie watermark data in 
accordance with on amplitude of tiie first set of 
frequency components of the intermediate sig- 
nal into which the watermark data is inserted. 

4. A waternrmrk data insertion method according to 
daim 1 , wherein tiie watermark data insertion step 
comprises determining the first set of frequency 
components of the intermediate signal into which 
the watermark data is inserted by using pseudo 
random numbers having frequency-dependent 
weights. 

5. A watermark data insertion method according to 
daim 1 , wherein tiie watermark data insertion step 
comprises: 

a weighting step for inaeasing or decreasing a 
signal amplitude of tiie watermark data in 
accordance with a frequency of tiie first set of 
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frequency components of the intermediate sig- 
nal into which the watermark data is inserted. 

6. A watermark data insertion method according to 
claim 1 , wherein the watermark data Insertion step s 
comprises converting binary data representing the 
watermark data into predetermined positive and 
negative values, and thereafter inserting the con- 
verted binary data into the first set of frequency 
components of the intermediate signal. io 

7. A watermark data insertion method for inserting 
watermark data into an input original signal, com- 
prising: 

15 

determining a position at which the watermark 

data is to be inserted in accordance with 

pseudo random numbers which have been 

subjected to an inverse transform; 

subjecting the watermark data to an inverse 20 

transform; 

increasing or decreasing a signal amplitude of 
the inverse-transformed watermark data in 
accordance with an amplitude of the original 
signal; and 25 
inserting the inverse-transformed watermark 
data with the increased or decreased signal 
amplitude into the original signal 

8. A watermark data detection method for detecting 30 
watermark data within a signal containing embed- 
ded watermark data, comprising: 



9. A watermark data detection method according to 
claim 8, wherein the averaging step comprises 



deriving the average of the signal components to 
which the watermark data has been added by 
applying a weight to an average of high frequency 
components among the signal components to 
which the watermark data has been added, the 
weight being smaller than weights applied to aver- 
ages of frequency components other than the high 
frequency components. 

10. A watermark data detection method for detecting 
watermark data within a signal containing embed- 
ded watermark data, comprising: 

a frequency transform step for applying a fre- 
quency transform to the signal containing 
embedded waternnark data to form an interme- 
diate signal; and 

a watermark data extraction step for extracting 
the watermark data from a predetermined fre- 
quency component of the intermediate signal. 

11. A watermark data detection method according to 
daim 10. wherein the watermark data extraction 
step comprises determining the predetermined fre- 
quency component of the intermediate signal from 
which the watermark data is extracted based on 
pseudo random numbers. 

12. A watennark data insertion apparatus for inserting 
watermark data into an input original signal, com- 
prising: 

a frequency transform section for applying a 
frequency transform to the original signal to 
form an intermediate signal; 
a watermark data insertion section for inserting 
the watermark data into a first set of frequency 
components of the intermediate signal; and 
a frequency inverse transform section for 
applying an inverse frequency transform to the 
intermediate signal, the watermark data having 
been inserted into the intermediate signal, 
thereby obtaining a signal containing the water- 
mark data embedded therein, 
wherein the watermark data insertion section 
determines the first set of frequency compo- 
nents of the intermediate signal into which the 
watern^rk data is inserted based on pseudo 
random numbers, and uses a second set of fre- 
quency components of the intermediate signal 
as a reference signal, the watermark data not 
being inserted into the second set of frequency 
components. 

1 3. A watermark data detection apparatus for detecting 
vratermark data within a signal containing embed- 
ded watermark data, comprising: 



an inverse-transformed pseudo random 
number generation step for applying an inverse 35 
frequency transform to pseudo random num- 
bers used for determining an insertion position 
of the watermark data and complements of the 
pseudo random numbers, respectively; 
a multiplication step for multiplying the signal 40 
containing embedded watermark dots by the 
values generated in the inverse-transformed 
pseudo random number generation step to 
obtain signal components to which the water- 
mark data has been added and signal compo- 45 
nents to which the waternr^rk data has not 
been added; and 

an averaging step for deriving a difference 
between an average of the signal components 
to which the watermark data has been added so 
and an average of the signal components to 
which the watermark data has not been added 
as obtained in the multiplication step, whereby 
the difference is detected as the waternrrark 
data. 55 
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an inverse-transformed pseudo random 
number generation section for applying an 
inverse frequency transform to pseudo random 
numbers used for determining an insertion 
position of the watermark data and comple- 5 
ments of the pseudo random numbers, respec- 
tively; 

a multiplication section for multiplying the sig- 
nal containing embedded watermark data by 
the values generated in the inverse-trans- 10 
formed pseudo random number generation 
section to obtain signal components to which 
the watermark data has been added and signal 
components to which the watermark data has 
not been added; and 15 
an averaging section for deriving a difference 
between an average of the signal conponents 
to which the watermark data has been added 
and an average of the signal components to 
which the watermark data has not been added 20 
as obtained by the multiplication section, 
whereby the difference is detected as the 
watermark data. 
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